Localisation and Augmented Reality for Mobile Applications in Culture Heritage

This study proposes to investigate augmented reality for historical tourism on mobile devices. The core of our proposition concerns a markerless outdoor augmented reality solution. Our approach for augmented reality is technically based on SIFT (Scale Invariant Feature Transform) features for localisation and integration of 3D models into video. These SIFT features are projected on a digital model of the building façades of the square to obtain 3D co-ordinates for each feature point. The algorithms implemented calculate the camera pose for frame of a video from 3D-2D point correspondences between features extracted in the current video frame and points in the reference dataset. The algorithms were successfully tested on video films of city squares. Although they do not yet operate in real-time, they are capable of a correct pose estimation and projection of artificial data into the scene. In case of a loss of track, the algorithms recover automatically. The study shows the potential of SIFT features for purely image based markerless outdoor augmented reality applications. The end-user goal of our application is to integrate into a lightweight and multimedia platform the tools required to visit a place and to obtain additional information about this place. * Corresponding author